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1.  Computable  Piecewise  Linear  Upper  and 
Lower  Optimal  Value  Bounds 

The  author  recently  proposed  a  simple  procedure  for  obtaining 
computable  global  upper  and  lower  parametric  bounds  on  the  optimal  value 
of  general  nonlinear  parametric  programs  for  which  overestimating  and 
underestimating  problems  can  be  constructed  that  have  either  a  convex  or 
concave  optimal  value  function.  The  class  of  programming  problems  that 
can  be  so  bounded  turns  out  to  be  extremely  large,  and  hence  the  potential 
applications  are  vast.  The  idea  is  based  on  properties  that  are  elementary 
and  well  known  but  apparently  have  not  been  previously  exploited  or  system¬ 
atically  developed  for  the  indicated  purpose. 

The  bounding  procedure  may  perhaps  be  most  easily  conveyed  geometri¬ 
cally,  for  problems  whose  optimal  value  function  is  convex.  Consider 
a  problem  of  the  form 


min^fCx.e.)  s.t.  g(x,e)  >  0,  h(x,e)  *  0 


P(e) 


where  f  is  real  valued  function,  g  and  h  are  vector  valued  functions,  I 

and  £  is  a  parameter  vector.  If  P(e)  is  jointly  convex,  i.e.,  if  f 
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is  jointly  convex  in  x  and  t.  ,  the  components  of  g  are  jointly  concave 
in  x  and  £  ,  and  those  of  h  jointly  affine  (linear)  in  x  and  c  , 
then  it  has  been  known  for  some  time  (Mangasarian  and  Rosen,  1964)  that 
the  optimal  value  function  f*(£)  of  P(c)  is  a  convex  function  of  (.  . 
Under  these  circumstances,  a  typical  graph  of  the  optimal  value  f*  of 
f  relative  to  a  given  component  of  c  with  fixed,  j  4-  i  , 

is  shown  in  Figure  1.1. 


y 


y-a 


2 


y=f. 


3 


t. . 

l 


FIGURE  1.1:  Convex  Optimal  Value  Function  and  Bounds 

Suppose  we  have  evaluated  f*(e)  and  its  slope  at  two  values  of 
,  where  the  component  =  0  and  e\  ,  and  (for  simplicity)  the 

other  components  of  s  fixed  at  0  .  Then,  under  general  circumstances, 
there  exist  the  supports  and  of  (the  epigraph  of)  f*(;  )  at 
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FIGURE  1.1:  Convex  Optimal  Value  Function  and  Bounds 


Suppose  we  have  evaluated  f*(c)  and  its  slope  at  two  values  of 
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_  ^ 

i. .  =  0  and  e.  =  e.  ,  respectively,  above  which  f  ft)  must  lie,  and 
1  11 

the  line  through  (0,f*(0)),  ,  below  which  l*(>  )  must 

lie,  so  that  f*(e)  must  lie  in  the  cross-hatched  triangular  area  indicated 
in  Figure  1.1  when  varies  between  0  and  .  If  t*(e)  is 

differentiable,  then  convexity  of  f*(s'.)  implies  that  f*(f  )  >  f*(c)  + 

V  f*(c)  (c-E)  from  which  it  follows  that 


Z2(e) 


f*(e)  + 


3f*(c) 

3c. 


(ei  -  ei> 


Hx(c) 


f*(0)  + 


3f*(Q) 

3e. 


e . 
i 


If  f*(e)  is  not  differentiable  then  any  subgradient  of  f*(0  at  C  , 
i.e.,  any  vector  v  such  that  f*(c)  >  f*(e)  +  vc(c-  c)  ,  would 
suffice  to  yield  a  global  linear  lower  bound  on  f*(c)  at  (f  ,  f*(c))  . 

In  particular,  we  can  use  the  directional  derivative  of  f*(0  at  any  point 

in  L lie  interior  of  its  domain,  since  this  is  known  to  exist  (Kockaf el lar , 
(1970).  (Obviously,  similar  constructs  apply  if  f*(c)  were  concave.) 

Thus,  in  the  interior  of  its  domain,  easily  computable  upper  and  lower 
bounds  on  f*(c)  for  any  finite  changes  in  c  can  be  provided,  by 
computing  two  solutions  of  Pic)  corresponding  to  two  values  of  c 
and  the  associated  supports  of  the  epigraph  of  f*(c)  at  the  selected 
values  of  the  parameters.  As  we  shall  see,  the  required  information  is 
provided  by  most  standard  NLP  algorithms  when  a  solution  has  been  deter¬ 
mined  for  the  given  parameter  values  and  the  associated  optimal  Lagrange 
multipliers  exist. 

11  the  given  problem  Pft_)  does  not  have  a  convex  optimal  function, 
then  we  can  still  generate  upper  or  lower  bounds  on  the  graph  of  f*(c) 
if  we  have  at  our  disposal  a  jointly  convex  or  concave  underestimating 
or  overestimating  problem  of  P(e)  ,  i.e.,  a  problem  whose  optimal  value 
must  underestimate  or  overestimate  f*(e)  for  the  parameter  values  of 
interest.  For  standard  (nonparametr ic)  NLP  programs,  there  exists  a 
variety  of  techniques  for  generating  convex  underestimating  problems  if 
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the  functions  of  the  given  nonconvex  problem  are  separable  (Falk,  and 
Soland,  1969)  or  factorable  (McCormick,  1976),  the  latter  class  including 
all  the  functions  we  have  encountered  in  practice. 

For  parametric  programs,  it  is  simply  a  matter  of  applying  these 
techniques  jointly  over  both  the  decision  variable  x  and  the  parameter 
t.  .  For  all  such  procedures,  the  calculation  of  bounds  of  the  form 
&1  or  ^  for  the  jointly  convex  underestimating  problem  will  provide 

piecewise-linear  lower  bounds  on  the  optimal  value  of  the  given  nonconvex 
problem.  This  technique  seems  especially  exploitable  by  branch  and  bound 
methods  that  are  based  on  solving  a  sequence  of  convex  underestimating 
problems. 

The  methodology  for  generating  convex  overestimating  problems 
is  less  developed,  though  some  results  exist.  Interior  penalty  (barrier) 
function  methods  (Fiacco  and  McCormick,  1968)  are  based  on  devising  an 
unconstrained  overestimating  problem  that  is  convex  if  the  problem  is 
convex  and  locally  convex,  otherwise.  Meyer  (1970)  developed  algorithms 
for  reverse  convex  constraints  based  on  constructing  a  sequence  of  linear 
overestimating  problems.  Avriel,  Dembo  and  Passy  (1975)  have  developed 
an  algorithm  for  solving  a  generalized  geometric  programming  problem, 
utilizing  a  sequence  of  posynomial  (hence,  equivalent  to  convex) 
overestimating  problems  in  conjunction  with  an  "interior"  cutting  plane 
method.  Chaemi  (1980)  has  derived  formulas  for  calculating  a  convex 
overestimating  problem  of  a  factorable  program.  If  a  jointly  convex 
overestimating  problem  can  be  constructed,  then  the  calculation  of  a 
bound  of  the  form  for  this  problem  will  provide  a  linear  parametric 

upper  bound  on  the  optimal  value  of  the  original  problem  over  the  range 
of  parameter  values  considered.  Analogous  bounds  can  obviously  be 
constructed  from  programs  whose  optimal  value  is  concave  and  some 
preliminary  results  are  also  reported  by  Ghaemi  (1980). 

This  procedure  for  calculating  optimal  value  bounds  was  first 
implemented  by  Fiacco  and  Ghaemi  and  reported  by  Ghaemi  (1980)  in  bis 
doctoral  dissertation.  Gh-'emi  added  the  bounds  calculation  capability 
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to  the  penalty-function  sensitivity-analysis  computer  program  SENSUMT 
and  used  this  program  to  successfully  obtain  bounds  for  several  small 
convex  and  nonconvex  problems.  Fiacco  and  Ghaemi  (1979a,  b)  applied 
the  program  to  calculate  bounds  on  the  optimal  value  of  a  convex  equiva¬ 
lent  of  a  geometric  programming  model  of  a  stream  water  pollution  abate¬ 
ment  system.  Fiacco  and  Ghaemi  (1980)  also  prepared  a  detailed  users' 
manual  for  SENSUMT  that  includes  the  optimal  value  bounds  calculation 
as.  a  user  option,  as  well  as  a  comprehensive  sensitivity  analysis 
capability. 

The  indicated  water  pollution  bounds  calculation  involves  the 
perturbation  of  a  single  right-hand-side  parameter,  the  allowable  oxygen 
deficit  level  in  the  final  reach  of  the  stream,  that  proved  unequivocably 
to  be  the  single  most  influential  parameter  in  the  prior  sensitivity 
study  by  Fiacco  and  Ghaemi  (1979b).  Subsequently,  Fiacco  and  Kyparisis 
(1981)  utilized  SENSUMT  to  calculate  bounds  on  the  optimal  value  function 
(the  annual  cost  of  operation)  when  30  (not  all  right  hand  side)  constraint 
parameters,  indicated  by  Fiacco  and  Ghaemi  (1979b)  to  be  the  most 
influential,  were  perturbed  simultaneously.  In  this  application,  the 
optimal  value  function  is  not  convex  in  a  full  neighborhood  of  the  base 
value  of  the  parameter  vector.  However,  it  was  possible  to  show  that 
the  restriction  of  f*(f.)  to  the  subset  of  parameters  involved  in  the 
desired  perturbation  is  convex. 

In  another  study  involving  the  convex  equivalent  of  a  geometric 
programming  model  of  a  power  system  energy  model,  to  find  the  turbine 
exhaust  annulus  and  condenser  system  design  that  minimizes  total  annual 
fixed  plus  operating  cost,  Fiacco  and  Ghaemi  (1981)  used  SENSUMT  to  obtain 
bounds  on  the  optimal  value  function  for  a  variety  of  single  objective 
function  and  constraint  parameter  changes.  A  novelty  of  this  analysis 
is  the  exploitation  of  problem  structure  to  calculate  a  nonlinear  lower 
bound  on  the  optimal  value  function. 

The  serious  implementation  of  the  bounds  procedure  for  separable 
and  factorable  nonconvex  programs,  implemented  for  some  small  examples 
by  Ghaemi  (1980),  remains  to  be  developed. 
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2.  Estimates  of  a  Parametric  Solution  Vector 
and  a  Sharper  Convex  Upper  Bound 

An  extremely  important  byproduct  of  this  bounds  calculation  is 
its  immediate  applicability  to  mul ti-parametric  perturbation  and  the 
availability  of  an  associated  feasible  parametric  vector.  Suppose  x(^) 

is  feasible  to  problem  P(e^)  and  x(e^)  solves  P(£j)  ,  while  x(£2) 

is  feasible  to  P(t2)  with  its  solution.  Assume  P(£)  is 

jointly  convex.  Define  c(a)  1  ac^  +  (l-a)E^  *  x(ot)  =  axft^)  + 

(1-a)  x  (e.)  ,  and  x*(a)  =  ax(e2)  +  ( 1— ot)  x(t^)  ,  where  0  ^  u  <  1  . 

It  can  easily  be  shown  that  x(ot)  (hence,  also  x*(u))  is  in  R[e(u)]  , 
for  any  0  <  a  <  1  ,  where  R (c)  is  the  feasible  region  of  P(f)  . 

Furthermore,  it  follows  that  f[x*(u)  ,  >.(a)  ]  is  a  convex  function  of 
a  that  lies  above  the  (unknown)  optimal  value  f*(c(a))  and  on  or  below 
the  known  linear  upper  bound  £^(e( u) )  of  f*(e(a))  ,  for  0  <  a  <  1  . 

These  facts  are  actually  involved  in  demonstrating  the  convexity 
of  f*(c)  and  its  domain  (Mangasarian  and  Rosen,  1964)  and  are  readily 
shown  as  follows.  Since  P(e)  is  jointly  convex,  it  follows  that 

g[x(a)  ,£(a)]  :  g[ax(e2l  +  (1-u)  x  (c^)  ,  +  (l-ujt^j 

>  ug[x(t;2)  ,  t_2  ]  +  (1-a)  g[x(<  :)  ,  t-1) 

>  0 

the  first  inequality  (meaning  that  the  inequality  holds  for  each  component 
of  g)  follows  from  joint  concavity  of  each  ,  and  the  second  from  the 

feasibility  of  xf^)  for  P(&1)  and  x(e2)  for  P(e2)  .  Since  the 

equality  constraints  are  jointly  affine  if  P(e)  is  jointly  convex, 
the  analogous  relations  hold  for  h[x(ot)  ,  e(a)J  with  the  inequalities 
replaced  by  equalities.  Hence,  x(a)  rR[c(ot)]  . 

-  6  - 
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If  f(x,t  )  is  jointly  convex,  l  lion 

f*U(u)J  <-  1  [x*(a)  ,  ,  (a)  | 

f  lax  (  l'  2 )  +  (1-u)  x(f  )  ,  liL*  ^  +  (l-a)i  jl 
<  af [x(c2),t:2J  +  (1-a)  f[x(r1)  .L.J 
-  af*(F2)  +  (1-a)  f*(c1) 

=  £3[e(a)] 

the  first  inequality  following  from  the  fact  that  x*(a)  r.  R[t(u)]  ,  which 
follows  from  the  preceding  result,  the  second  from  the  convexity  assumption. 
The  convexity  of  f  in  (x,  e)  and  the  fact  that  (x*  ,  e.)  is  linear  affine 
in  a  ,  implies  f[x*(a)  ,c(a)]  is  convex  in  a  .  Thus,  the  stated  results 
hold . 

The  depiction  of  f*(e(cx))  and  its  bounds  is  shown  in  Figure  2.1. 

This  figure  is  comparable  to  Figure  1.1.  We  have  also  depicted  the  ith 
component  of  x*(a)  e  R[c(a)]  in  the  figure.  Note  that  the  assumption 
that  P(C)  be  jointly  convex  can  be  relaxed.  It  suffices  that  P[t(a)] 
be  jointly  convex  in  x  and  a  to  obtain  these  results  and  this  depiction 
for  e  =  e(u)  . 


y=x£(u) 

y=f [x* (a) ,  c(u)]_ 


1 


) 


-  .  ■  y 

y-f*(e(a)/^  / 


/  y=f-i 


T-i 


FlGURIi  2.1:  Bounds  on  the  optimal  value  f*(c(u))  and  the  ith 
component  of  the  feasible  vector  x*(a)  ,  when  — 
problem  is  solved  for  a  “  0,1.  All  solid  lines 
represent  computable  quantities. 
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This  remarkably  regular  behavior  is  exploitable  in  many  ways. 

Since  x*(°0  is  the  result  of  linear  interpolation  between  two  given 
solutions,  if  additional  solutions  are  obtained  corresponding  to  interme¬ 
diate  values  of  a  ,  0  <  a  <  1  ,  then  it  is  apparent  that  a  piecewise- 

Linear  feasible  parametric  vector  x('t)  can  be  obtained  by  linear 
interpolation  between  contiguous  solutions.  Further,  the  function 
i;:(i-(a))  will  lie  between  piecewise-linear  continuous  upper  and  lower 
parametric  bounds  that  can  be  readily  computed,  and  that  will  become 
tighter  with  eacTi  new  intermediate  solution  that  is  computed,  e.g., 
at  a  value  of  a  where  the  deviation  between  the  upper  and  lower  bounds 
is  greatest.  Hence,  if  a  prescribed  error  tolerance  in  estimating 
f*(i(a))  for  0  <  a  <  1  is  given,  then  we  need  only  solve  P(‘('0) 

tor  several  values  of  a  in  [0,1]  ,  until  the  bounds  on  f*(l  ('<■)) 
do  not  deviate  by  more  than  the  given  tolerance. 

It  follows  from  what  was  shown  that  the  objective  function  f(x  ,t) 

along  (x(a)  ,c(ot))  is  piecewise-convex,  does  not  underestimate  the  actual 
optimal  value  function  f*(C(a))  ,  and  has  its  graph  on  or  below  the 
known  piecewise-linear  upper  bound,  over  the  specified  range  of  ft 
Thus,  with  every  solution  corresponding  to  an  intermediate  value  of  a  , 
we  may  easily  calculate,  in  addition  to  better  piecewise-linear  contin¬ 
uous  upper  and  T.ower  bounds,  a  more  accurate  piecewise-linear  continuous 
leasible  estimate  of  the  parametric  solution  vector  along  with  a  more 
accurate  piecewise-convex  continuous  over-estimate  of  the  optimal  value 

limit  ion.  We  illustrate  the  parametric  optimal  value  bounds  and  associated 
leasible  vector  solution  estimate  available  after  solutions  have  been 
obtained  for  three  value  of  a  ,  in  Figure  2.2. 

This  provides  a  general  approach  [or  generating  optimal  value 
bounds  and  an  associated  parametric  solution  vector  estimate  for  general 
nonlinear  parametric  problems.  Most  of  these  concepts  are  well  known  and 
simple,  the  essential  novelty  being  their  exploitation,  adaptation  and 
tailoring  to  the  purpose  at  hand.  The  calculations  involve  the  manipula¬ 
tion  of  information  generally  already  provided  by  standard  solution 
algorithms. 

There  are.  however,  numerous  important  theoretical  and  algorithmic 
retinements  that  must  be  made  to  shape  these  techniques  into  a  coherent. 
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FIGURE  2.2:  Bounds  on  the  optimal  value  f*{e(a)J  and 
the  1th  component  of  the  feasible  vector 
x*(a)  ,  when  problem  is  solved  for  a  =  0,  a  ,  1. 
All  solid  lines  represent  computable  quantities. 
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incisive  and  efficient  methodology.  Their  development,  validation,  and 
computational  implementation  is  being  pursued.  hut  it  already  appears 
plausible  that  an  abundance  of  sensitivity  and  stability  information 
can  be  calculated  for  any  solvable  NLP  problem,  with  relatively  little 
additional  computational  effort.  The  kind  of  information  that  can  be 
calculated  begins  to  be  somewhat  comparable  in  scope  and  detail  to  the 
essential  post-opt imality  parametric  sensitivity  and  range  analysis 
information  that  is  heavily  utilized  in  linear  programming.  In  fact, 
it  seems  very  likely  that  NLP  stability  results  will  bring  new  techniques 
for  estimating  the  effects  of  large  perturbations  in  linear  programming 
us  well. 

Many  research  directions  require  exploration  in  developing  the 
proposed  bounds  techniques.  We  mention  a  few:  (1)  systematic  accumula¬ 
tion  of  computational  experience  with  moderate-sized  jointly  convex 
programs,  allowing  for  the  simultaneous  perturbation  of  several  parameters 
(1!)  development  of  the  optimal  value  bounding  procedure  for  separable  and 
tadorable  programs,  based  on  known  procedures  for  formulating  a  jointly 
convex  underestimating  problem  for  the  given  parametric  nonconvex  problem, 
applying  the  bounding  technique  to  the  convex  problem,  and  branch  and 
bound  and  other  techniques  for  improving  accuracy;  (I)  a  deeper  analysis 
ol  conditions  on  the  problem  functions  that  are  sufficient  for  the 
convexity  or  concavity  of  the  optimal  value  function  for  specified, 

(often  simple,  e.g.,  vector)  perturbations  of  problem  parameters, 
allowing  for  the  applicability  of  the  optimal  value  bounding  procedure 
for  nonconvex  problems  without  requiring  the  construction  of  convex  under¬ 
estimating  or  overestimating  problems;  (4)  continued  demonstrations  of 
practical  applicability;  and  (5)  the  further  development  of  computational 
techniques  that  exploit  standard  NLP  algorithm  calculations,  analogous 
to  those  introduced  by  the  author  for  penalty  function  algorithms,  thus, 
hopefully  stimulating  the  routine  and  widespread  use  of  sensitivity  and 
stability  information  for  all  standard  NLP  algorithms  just  as  in  linear 
programming. 
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j.  Connections  Between  Optimal  Value  Bounds  and  Duality 

The  foregoing  optimal  value  bounds  results  have  many  points  ol 
contact  with  duality  theory.  In  fact,  the  bounding  procedure  and  its 
computability  were  initially  suggested  to  the  author  by  the  following 
well-known  connections  with  optimality  conditions  and  duality  results. 

Suppose  the  problem  is  a  right-hand-side  perturbation  problem  of  the 
form 

min^  f(x)  s.t.  g(x)  >'■  P1(r) 

where  xCEn  ,  g  is  a  m  component  vector  function  and  t-  is  in  £m  . 

An  immediate  connection  is  well  known:  if  tne  karus'n-kuhn-Tucker  cone. i >.  ior.s 

iioiu  at  (x(c),  u(i:))  ana  if  (x(c),  u(t))  is  unicue  and  differentiable 

near  c  =  0  ,  then  3t*(f.)/3c^  =  u^(c)  near  c  =  0  (Armacost  and  Fiacco, 

197b).  Suppose  also  that  P^(c)  is  convex  in  x  and  for  simplicity 

assume  tliat  only  one  parameter  t  ^  is  perturbed,  that  k  >  0  ,  that  a 

solution  x(i.)  of  Pj(i)  exists  J  or  all  ^  satisfying  the  Karush- 

Kunh-luckcr  conditions,  and  that  glx(i)J  -  f  •  Then,  it  follows  that 

the  optimal  value  f*(c)  is  convex  and  nondecreasing  as  a  function  of 
s.  ,  as  in  Figure  1.1.  At  the  solutions  x(0)  and  x(r)  ,  where 

f  .  =  c .  and  t  =  0  ,  j  f  i  ,  the  lower  bounds  1,  and  as  depicted 

li  J  12 

in  figure  1.1  apply,  and  have  nonnegative  slopes  corresponding  to  non- 

1  ? 

negative  optimal  Lagrange  multipliers,  say  uj  and  uf  ,  respectively. 

Clearly,  we  obtain  )  =  f*(0)  +  u!r.,  and  ^(t )  =  f*(c)  +  u?(£^  -  C..)  . 

It  is  clear  from  Figure  I. 1  that  any  such  tangent  to  the  graph  of  f * ( C ) 
wil  l  imersect  the  y-axis  at  or  below  the  optimal  value  f*(c)  at  t  =  0. 

In  fact,  it  may  be  inferred  from  the  assumptions  and  the  figure 

that 
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1 

U  .  €  .  = 

1  1 

f*(0) 

in 

-  I  u.(O)g.fxlO))  =  L l x ( 0  3  , 

,  u ( 0 )  ,  01 

i-l 

2 

U.E  .  = 

1  1 

m 

f*(C) 

-  u.(,  )g.(x(,  ))  =  L l x (!  )  , 

.  n(t  )  ,  ol 

where  x(0)  solves  P^(0)  with  associated  optimal  Lagrange  multiplier 

u(0)  such  that  u^(0)  =  u^  and  x(>  )  solves  lJ^(t  )  with  associated 

optimal  Lagrange  multiplier  u(e)  such  that  u^(f)  =  ujr  ,  L  denoting 

the  usual  Lagrangian  associated  with  Pj  (0)  .  Hence,  the  4.  and 

y-intercepts  are  L[x(0)  ,  u(0)  ,  0]  and  L[x(e)  ,  u(i  )  ,  0]  ,  respec¬ 
tively.  It  is  obvious  from  the  figure  and  the  assumptions  that 
£*(0)  =*  L[x(0)  ,  u(0)  ,  0]  and  accordingly  that  x(0)  solves  P^(0)  . 

Uual  problems  involved  the  determination  of  the  slope  (Lagrange 
multiplier)  ol  a  supporting  line  (or  plane,  in  general)  like  t.  such 

that  the  minimal  Lagrangian  value  over  x  (i.e.,  the  y-intercept)  is 
maximized.  For  example,  the  Wolfe  dual  (Wolfe,  1961)  of  the  convex 
program 


min  f(x)  s.t.  g(x)  >  0 


(P) 


is  given  by 

max  L(x  ,  u)  s.t.  V  L(x  ,  u)  =  0  ,  u  >  0  ,  (D) 

(x.u)  X 

where  L(x  ,  u)  =  f(x)  -  Uu  g.(x)  ,  the  usual  Lagrangian.  It  is  easily 
shown  that 

inf  f(x)  >  L(x  ,  u)  (3.2) 

K  ~ 


where  R  is  the  feasible  region  of  problem  P  and  (x  ,  u)  belongs  to 
the  feasible  set  RQ  of  D  .  Under  appropriate  regularity  conditions. 
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it  also  follows  that 


minDf(x)  =  max  L(y  ,  u) 

R  kD 

and  these  conditions  essentially  summarize  the  duality  relationship. 

To  establish  a  connection  between  this  duality  concept  and  the 
hounds  discussed  earlier  and  depicted  in  Figure  1.1,  consider  the  Wolfe 
dual  of  P|(t.)  ,  given  by 

max  L(x  ,  u  ,  e)  s.t.  V  L(x  ,  u  ,  c)  =  0  ,  u  >  0  D  (£) 
(x,u)  *  "  1 

where  L(x  ,  u  ,  e)  =  f(x)  -  uT[g(x)  -  e]  and  V  L  =  V  f  -  uTV  g  . 

xx  x 

For  simplicity,  suppose  P^(e)  has  a  solution  for  any  €  .  Two  important 

properties  of  D^(l)  should  be  noted:  (i)  the  constraints  of  D^(e) 

fill  not  depend  on  L  and  (ii)  the  duality  results  just  given  imply  that 
Ltie  optimal  value  f*(£)  of  P^(£)  is  bounded  below  by  the  objective 
function  of  D j ( £ )  at  any  feasible  point  of  0^(0)  »  i.e., 

f*(£)  >  L(x  ,  u  ,  e)  (3.3) 

=  f(£)  -  aT[g(x)  -  £] 

=  uTc  +  L(x  ,  u  ,  0) 

for  every  (x  ,  u)  that  satisfies  V^L  =  0  and  u  >  0  .  This  generalizes 

the  duality  result  (3.2)  just  given  (which  apply  for  t  =  0  ,  essentially) 
and  provides  immediate  connections  and  extensions  of  the  bounds  results 
given  earlier. 

Dual  feasible  points  of  P^(£)  may  be  found  by  solving  perturba¬ 
tions  of  P^(0)  .  Suppose  the  problem  P^(C)  is  solved  at  t  by 
x(~)  .  Then,  under  appropriate  regularity  conditions  these  exists 
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u(c)  >  0  such  that  L[x(  1  ,  u(.  1  ,  •.  ]  =  0  ,  hence  ixt,  1  ,  u(:  )) 

is  dual-feasible,  and  funlier,  m-  ) 1  1  l ^ >  )1  -  ;  |  =  0  .  We  note  in 
passing  that  if  these  Karush-Kuhn-Tueker  conditions  hold,  then 

f*(i  )  =  L[x(t)  ,  u(u)  ,  £.]  and  hence  the  above  relations  imply  that 

(x(c)  ,  u(l)>  solves  the  dual  l)^(r)  of  r ^ ( >  )  .  Returning  to  the 

bounds  relationship,  we  observe  that  [x(e)  ,  u(c)]  is  a  feasible  point 
of  the  dual  D^(e)  of  P^(i)  for  any  f.  .  Thus,  from  (3.3)  we  conclude 

that 


f*(e)  >  L[x(f!)  ,  u(D  ,  c]  =  f[x<c)  j  -  u(e)T(t  -  l) 

=  u(C)T  t  +  L[x(C)  ,  Ufr)  ,  0]  .  (3.4) 

As  observed,  under  appropriate  regularity  assumptions,  this  hound  is  met 
when  <  =  i  ,  with  the  inequality  generally  being  strict  for  all  <.  f  >. 
in  the  domain  of  f *(t  )  . 

lL  is  clear  that  if  P^(l)  satisfies  the  conditions  imposed  at  the 

beginning  of  this  section  then  the  lower  bound  on  f*(1^)  that  we  have 
just  obtained  in  (3.4)  is  the  same  as  the  bound  1 ^  in  Figure  1.1  that 

we  previously  calculated  in  (3.1). The  bound  (3.4)  is  more  general  in 
allowing  the  perturbation  of  the  entire  parameter  vector  i  ,  rather 
than  only  one  component  as  in  (3.1)  and  Figure  1.1.  Of  course,  the 
correspondence  with  the  bound  X.  in  Figure  i.l  and  (3.1)  is  also 

established  as  above,  replacing  >.  by  0  .  Summar izing,  it  follows 

that  a  linear  lower  bound  on  the  optimal  value  of  1*0  )  of  PjO)  at  1  , 

supporting  the  set  of  (y,t)  such  that  y  >  f*(x)  ,  corresponds  to  the 

Lagrangian  Wolfe  dual  value  of  P^(t:),  where  x  is  evaluated  at  a  solution 

x(t)  of  P  (r.)  for  r.  *  t.  ,  with  u  equal  to  the  associated  optimal 
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Lagrange  multiplier  u(c)  .  Assuming  (x(e)  ,  u(£))  satisfies  the 
Karush-Kuhn-Tucker  conditions,  it  follows  that  f*(c)  =  L[x(g)  ,  u(c)  ,  t] 

and  (x(e)  ,  u(c))  is  a  Wolfe  dual  feasible  point  of  P^(e)  for  all  £ 

in  the  domain  of  f*(e)  . 

4.  Nonlinear  Dual  Lower  Bounds 

This  duality  interpretation  .lead’s  to  further  extensions  of  the 
bounds  results.  Observe  again  th .*t  Che  two  essential  properties  of  the 
dual  l)^(e)  that  lead  to  computable  lower  bounds  on  f*(>-)  are  the  dual- 
lower  bound  (3.3)  and  the  lack,  of  dependence  of  the  constraints  of  D^(;  ) 

on  l  .  This  permits  the  calcuiai  Jan  of  the  parametric  lower  bound  (3.3) 

whenever  any  feasible  point  (x,u)  of  D^e)  has  been  computed.  But 

these  conditions  and  resuits  also  hold  for  the  dual  D^e)  of  the  more 

general  parametric  problem 

min  f(x)  s.t.  g(x)  >  g(e)  P^c) 

'V/ 

where  g(i )  is  an  m-component  vector  function. 

Assuming  l’^(e)  is  convex  in  x  ,  the  Wolfe  dual  is 

f\J 

‘nafx,u)L*X,u,e)  S,C'  vxL<x.u»e)  *  0  »  u  >  0  Di(t) 

a. 

Feasible  points  (x,u)  of  D^(e)  again  do  not  depend  on  z  and  a  bound 

of  the  form  (3.3)  may  be  calculated  analogously,  yielding  for  any  dual 

%  %  ■v 

feasible  point  (x,u)  of  D^(c)  the  inequality 

f*(e)  >  L(x,u,f.)  -  f(x)  -  uC[g(x)  -  g(£)J  (4.1) 

'V  *v>  'v 

*  u  g(e)  +  L(x,u,o) 
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in  terms  of  the  constituents  of  P^(c)  and  D^(e)  .  In  particular, 
analogous  to  the  previous  treatment  with  respect  to  P^(e)  and 
that  gives  (3.4),  we  conclude  from  (4.1)  that 

f*('-)  >  L[x(t),  u(t)d  1  =  u(t:)t'  g(t  )  +  L|x(>),  u(t),  <>l  (4.2) 

where  £*  and  L  now  refer  to  the  objective  functions  of  P^  and  , 

'X, 

respectively,  and  x(£)  now  solves  P^(£)  with  associated  optimal  Lagrange 

multiplier  u(e)  .  If  the  Karush-Kuhn-Tucker  conditions  hold,  then  the 

_  t  -v  _  % 

r.h.s.of  (4.2)  becomes  f[x(e)]  -  u(c)  [g(e)  -  g ( c ) ]  and  of  course 

(4.2)  holds  with  equality  when  e  =  c  .  The  bounds  given  by  (4.1)  and 
(4.2)  generalize  (3.3)  and  (3.4).  They  are  global  (i.e.,  valid  for  all 
c  in  the  domain  of  f*(e)),  allow  for  the  perturbation  of  all  components 

% 

of  l.  ,  and  do  not  require  any  convexity  restrictions  on  g(>_)  . 

% 

ft  is  noted,  however,  that  il  the  components  of  g(e.)  are  convex 
or  concave,  then  the  (more  general)  lower  bound  (4.1)  is  accordingly  convex 
or  concave.  In  the  former  case,  the  Lagrangian  (dual)  lower  bound  on 
f*(t:)  will  obviously  coincide  with  or  be  sharper  than  the  bound  derived 
from  its  iinearization  (as  in  Figure  1.1)  at  any  value  of  e  in  the  domain 

of  f*(>  )  ,  over  any  subset  of  the  domain  of  f*(c)  .  In  the  latter  case, 
the  linear  estimate  of  this  Lagrangian  dual  bound  at  any  parameter  in  the 
domain  would  yield  a  parametric  upper  bound  on  the  dual  value,  while 
linear  interpolation  between  two  parameter  values  would  yield  a  linear 
lower  bound  on  (4.1)  and  hence  on  f*(t)  . 

It  should  be  noted  that  the  duality  results  given  for  P  (s.) 

extend  in  an  obvious  manner  to  the  more  general  inequality-equality 
constrained  right-hand-side  convex  program  of  the  form 

'V  rv 

min^  f(x)  s.t.  g(x)  >  g(f  ),  h(x)  =  h(s.)  !'.,(•  ) 
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where  h(x)  is  linear  affine  in  x  ,  recalling  that  the  Wolfe-dual  of 
P2(c')  is 

max  L(x,u,w,£)  s.t.  V  L(x,u,w,c)  =  0  ,  n  >  0  n7('  ) 

(x,u,w) 

wh  re  L  now  denotes  the  Lagrangian 

L(x,u,w,t)  f(x)  -  uC[g(x)  -  g(c)]  +  wC  fh(x)  -  h(i.)  j 

a, 

o.  P  (s'.)  .  Again,  the  constraints  of  D  (t)  do  not  depend  on  ■_  and 
the  !>ound  of  cite  torr.i  (4.2)  applies. 

The  fact  that  no  convexity  (or  concavity)  restrictions  art  needed 
on  g(  or  h(  )  to  calculate  bounds  may  great lv  fat  i 1  '  the  calcuJa- 

\i 

tion  at  useful  bounds  for  programs  ot  the  font  P  ^ (  )  or  P?(c)  that 
are  not  convex  in  x  .  in  particular,  it  suffices  to  obtain  a  convex 

f'j  ■> 

underestimating  problem  "in  x"  ,  i.e.,  without  altering  g(')  ft  h(.-  )  . 
The  duality  results  just  given  can  be  applied  to  this  problem,  yielding 
a  lower  bound  of  the  form  (4.1)  on  its  objective  function  value,  hence 
u  the  optimal  value  of  the  original  problem,  whenever  a  dual  feasible 
pc  Ln^.  of  the  underestimating  problem  has  been  calculated. 

It  is  stressed  that  it  is  not  necessary  Lo  solve  tile  primal 

problem  P  ( > )  or  p  ( .)  for  some  value  of  in  t  he  domain  of 

interest  to  obtain  a  lower  bound  on  .  it  suit  ices  to  obtain  a»v 

lc  i  1  -1  e.is  Lble  point,  to  evaluate  the  parametric  lower  bound  given  in  (4.1)  . 
ibis  fact  is  exploitable  by  any  NLP  algorithm  that  calculates  a  sequence 
of  dual  feasible  points  in  approaching  a  solution.  In  particular,  the 
penalty  function  calculations  that  we  have  heavily  utilized  for  the 
a  1  gov ithmic  estimation  of  sensitivity  analysis  (Fiacco,  197b)  also  pro¬ 
vides  an  excellent  example  of  this  basic  idea  of  an  iterative  algorithmic 
.  proximation  of  optimal  value  bounds. 


17 


1-45 1 


For  example,  If  a  sequence  of  dual  feasible  points  f(x^(t),  u^(f)))  is 

a,  _ 

generated  for  D^(e)  when  k  =*  1,2,...  by  the  given  solution  algorithm 

a, 

in  the  process  of  solving  P^e),  as  is  characteristically  true  of  penalty 

functions  (Fiacco  and  McCormick,  1968),  then  we  could  conclude  from  (4.1) 
that  for  each  k 

—  t  —  — 

f*(t)  >  uk(e)  g(t_)  +  L[xk(c)  ,  uk(c)  ,  OJ 

% 

over  the  domain  of  f*(e)  ,  providing  P^(c)  is  convex  in  x  .  (The 

inequality  applies  even  if  c  is  replaced  by  ck  ,  i.e.,  even  if  C 

changes  for  each  k  .)  Suppose  {(xk(t)  ,  uk(e))}  converges  to 

(x(l),  u (r.))  ,  where  x(e)  solves  P^(c)  with  associated  dual  multiplier 

u(c )  ,  as  generally  happens  for  penalty  function  dual-feasible  convergent 
subsequences.  Then  of  course  the  above  bound  approaches  f*(e)  >  u(e)t 
%  _ 

g(c)  +  L[x(e),  u(e),  0]  if  the  problem  functions  are  continuous.  If 

(x(e),  u(c))  satisfies  the  Karush-Kuhn-Tucker  conditions,  then  the  bound 
is  satisfied  exactly  at  C  . 

The  dual  problem  and  a  resulting  lower  bound  can  also  be  given  for 
the  general  parametric  program  P(e)  that  is  convex  in  x  .  The  problem 
and  its  dual  are 

min  f(x,£)  s.t.  g(x,e)>  0  ,  h(x,t)  *  0  P(£) 

X 

and 

max  ,L(x,u,w,e)  s.t.  V  L(x,u,w,£)  =  0  ,  u  >  0  D(£) 

\  x ,  U  ,  W  )  X 

where  now  L(x,u,w,c)  =  f(x,c)  -  utg(x,e)  +  wth(x,e)  .  As  before,  the 
optimal  value  of  P(t)  is  bounded  below  by  any  dual-feasible  value  of 
L(x,u,e)  ,  i.e., 

f*(>)  >  L[x(e),  u(c),  w(e),  e]  , 
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for  any  point  (x,  u,  w)  satisfying  the  dual  constraints.  The  difference 

now  is  that  the  dual-feasible  point  (x,  u,  w)  generally  depends  on  t 
and  may  not  be  computable,  even  over  a  small  subset  of  the  domain  of  t: 
of  interest.  Nonetheless,  as  noted  in  Section  1,  piecewise  linear  upper 
and  lower  bounds  can  be  calculated  if  f*(e)  is  convex.  The  linear 
supports  and  £0  for  P(e)  were  derived  based  solely  on  the  convex¬ 

ity  assumption.  A  duality  interpretation  can  again  be  given,  however,  when 
P(e)  is  jointly  convex.  Motivation  for  this  is  the  idea  that  we  may  be 
able  to  characterize  any  linear  lower  bound  such  as  given  in  Figure  1.1 
as  the  graph  of  the  optimal  value  associated  with  some  parametric  linear 
program.  The  convexity  of  f*(e)  and  its  relationship  to  the  Lagrangian 
support  this  idea  and  in  fact  immediately  yield  an  equivalent  Lagrangian 
bound.  Suppose  (x(e),  u(e),  w(t))  satisfies  the  Karush-Kuhn-Tucker 
conditions  and  f*(e)  is  differentiable.  These  conditions  and  the 
following  consequences  hold  under  well-known  assumptions  (Armacost  and 
Fiaeeo(197  5) ,  Fiacco  (1976)]:  f*(e)  =  L[x(e),  u(e),  w(t),t;]  and 
V,-f*(e)  =  V't:L[x(e),  u(e),  w(e),  c]  .  Since  f*(t)  is  convex,  it  follows 

that  f*(e)  >  f * ( c)  +  Vt_f*(e)  (r.-E)  ,  for  c  ,  E  in  the  domain  of  f*(e)  . 

The  given  relationships  imply  that  this  convexity  condition  is  equivalent 

to  f *(e)  =  L [ x ( c ) ,  u(e) ,  w(c),  e]  >  L[x(E),  u(E),  w(E),  E]  + 

V..L[x(e),  u(E),  w(c),  t]  (e-  E)  .  The  given  lower  bound  on  f*(e)  is 

precisely  analogous  to  the  linear  lower  bounds  derived  earlier  and  suggests 
the  appropriate  J inear  program  and  duality  interpretation. 

Consider  the  linear  program  derived  from  the  jointly  convex  program 
P(<:)  by  linearizing  the  objective  function  and  binding  constraints  about 
(x(e),  E)  ,  thus  yielding  (with  nonbinding  constraints  deleted) 

minx  f[x(e),  c]  +  Vxf[x(e),  e]  [x-x(E)  ]  +  V.f  (x(E) ,  E]  (e-  E)  LP(e) 

s.t.  g[x(f),  c]  +  V’xg[x(e),  c]  [x-x(c)  ]+  Vf.glx(e) ,  c]  (c-  E)  >0 

h[x(£),  t:]  +  Vxh[x(E),  E]  [x-x(e)  ]  +  V  h[x(c) ,  E]  (c-  E)  «  0 
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Since  P(t)  is  jointly  convex,  it  foLlows  that  l.P(:  )  is  .in  under¬ 
estimating  problem  for  P(c)  ,  i.e.,  ?*(£)  v  f*(l  )  ,  where  f*(t  )  is 

the  optimal  value  of  LP(f.)  .  The  Wolfe  dual  of  !.P(c)  is  given  by 

max  f  +  Vxf(x-x)  +  V^f (e-  c)  -  ut[g  +  Vxg  (x-x)  +  VL.g(c-  c) 

(x,u,w) 

+  wt[h  +  V  h(x-x)  +  V^h(t-  Z)  DLP(e) 

s.t.  Vf-utVg+wtVh=0,u>0 
X  X  x  - 

where  the  superbar  denotes  evaluation  at  c  .  This  reduces  to 

max  L  +  V  L(£-  £)  s.t.  V  L  *  0  ,  u  >  0  DLP(e) 

(x,  u, w) 

Thus,  we  deduce  from  the  duality  relationship  that 
£*(e)  >  f*(e)  >  L  +  V  L  (t-  c) 

which  corresponds  precisely  to  the  bound  derived  just  previously,  resulting 
simply  from  the  convexity  of  f*(e)  . 

Summarizing,  for  the  general  jointly  convex  program  a  linear 
lower  bound  on  f*(c)  is  provided  by  the  dual  value  of  the  linearization 

of  P(e)  at  some  (x,  e)  ,  where  x  solves  P(e)  with  optimal  multipliers 

(u,  w)  . 


5 .  further  Extensions,  Recapitulation,  and  Future  Research 

Apart  from  duality  results  and  the  exploitation  of  convexity,  we 
note  that  other  problem  structures  may  allow  the  calculation  of  bounds. 
For  example,  suppose  the  problem  is  P(t):  min  f(x,c)  s.t.  x  e  R(e)  , 
where  R(c)  is  a  convex  point  to  set  map,  as  it  is  in  the  jointly  convex 
program,  but  f(x,e)  has  no  particular  convexity  structure.  We  can  still 
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calculate  a  parametric  upper  bound  on  f*[e(cO]  for  t.(a)  *  tx  r„  + 

(1-a)  £^  ,  when  £^  and  are  in  a  convex  subset  of  the  domain  of 

f*(c)  and  0  <  a  <  1  ,  once  x(e^)  £  R(r^)  and  x  (£ £  R(c^)  are 

available,  since  the  convexity  of  the  map  K(c)  implies  as  observed 
in  Section  2  that  x(a)  =  a  x(£^)  +  (1-  u)  x  (l^  )  l  R[t.  (u)  ]  for 

0  <  u  <  1  .  It  follows  that  f*[c(ot)]  <  f[x(<),  £(  «))  tor  all  0  <  a  <  1 

Another  class  of  problems,  one  not  requiring  convexity  in  x  , 

for  which  both  upper  and  lower  bounds  on  f*(e)  can  be  calculated,  are 

problems  of  the  form 

min  f(x,£)  s.t.  x£R 

where  R  does  not  depend  on  £  .  If  f (x,c)  is  concave  in  £  ,  then 
it  is  well  known  that  f*(c)  is  concave.  This  allows  for  the  calculation 
of  piecewise-linear  upper  and  lower  global  bounds,  using  linear  supports 
to  provide  upper  bounds  and  linear  interpolation  to  provide  lower  bounds 
on  f*(£)  ,  analogous  to  the  bounding  procedure  that  was  discussed  at  the 
beginning  of  this  section  for  the  convex  optimal  value  function. 

The  techniques  described  here  provide  an  approach  for  obtaining 
computable  optimal  value  bounds  and  an  associated  solution  vector  for 
large  classes  of  parametric  nonlinear  programs.  A  basis  for  obtainii^ 
upper  and  lower  bounds  on  the  optimal  value  is  the  convexity  or  concavity 
of  the  optimal  value  function  of  the  problem  of  interest,  or  of  an 
auxiliary  problem  that  underestimates  or  overestimates  the  optimal  value 
of  the  given  problem.  The  derivation  of  an  associated  feasible  parametric 
vector  function  is  an  immediate  consequence  of  the  convexity  of  the  point 
to  set  map  that  defines  the  feasible  region  R(l)  as  a  function  of  the 
parameter  c  . 

The  bounding  approach  has  direct  connections  with  duality  theory, 
as  noted,  leading  to  further  results  for  lower  bounds.  Namely,  the 
convexity  of  the  given  problem  in  the  variables  becomes  exploitable, 
giving  a  computable  procedure  for  problems  involving  general  right-hand- 
side  perturbations.  The  application  of  these  techniques  to  nonconvex 
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programs  to  obtain  global  parametric  Lower  bounds  via  conv>'<  under¬ 
estimating  problems  is  immediate  and  leads  to  the  idea  of  exploiting 
nonconvex  solution  methodologies  based  on  convex  underestimating  prob¬ 
lems,  e.g.,  branch  and  bound  methods  for  calculating  global  solutions. 
This  approach  remains  to  be  developed.  Other  directions  of  research 
that  should  be  carefully  explored  are  the  identification  of  additional 
classes  of  problems  that  have  convex  or  concave  optimal  value  functions, 
the  further  exploitation  of  problem  structure,  such  as  separability,  to 
construct  related  computationally  tractable  programs  with  convex  or 
concave  optimal  value  functions,  the  use  of  other  convex  and  nonconvex 
duality  results  (e.g.,  Rockafellar  (1970,  1974))  and  the  development  of 
bounds  on  the  parametric  solution  vector  and  the  associated  optimal 
multipliers.  Results  based  on  weaker  conditions  implying  any  exploit¬ 
able  property  of  the  optimal  value  function,  e.g.,  quasi-  or  pseudo¬ 
convexity,  would  be  extremely  useful  to  accommodate  larger  classes  of 
problems  and  remain  to  be  developed. 

The  reader  interested  in  interpretations  associated  with  Figure 
1.1  is  also  referred  to  Lasdon  (1970)  and  Bazaraa  and  Shetty  (1979). 

6.  Bounds  on  a  Solution  Point 


We  turn  briefly  to  the  problem  of  constructing  computable  bounds 
on  a  solution  point  x(e)  of  the  general  problem 

minxf(x,e)  s.t.  g(x,e)  >  0  ,  h(x,e)  =  0  P(e) 

Suppose  P(e)  is  jointly  convex,  x(e^)  solves  P(e^)  and  x(C2) 
solves  P(£2)  •  Recalling  the  notation  and  results  of  Section  2,  it  was 
concluded  that  x*(a)  £  R[e(a)  ]  for  any  0  <  a  <  1  ,  where  e(a)  *  + 

(l-a)e^  and  x*(a)  -  ax^)  +  (!“<*)  x  (k-^)  •  Furthermore,  we  found  that 
max[2,^(a),  j^Ca)]  <  f*[e(a)]  <  f  [x*(a)  ,e(a)  ]  .  Thus,  given  any  value  of 
a  g  1 0 , 1  ]  ,  we  are  able  to  calculate  a  feasible  point  x*(°0  of  p [ e (ot)  ] 
and  upper  and  lower  bounds  on  the  optimal  value  f * [ e (ot)  ]  of  P[e(a)]  , 
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once  solutions  of  P [ c ( 0) ]  and  V [t  (1) ]  are  available.  We  address 
the  problem  of  estimating  the  distance  of  x*(u)  to  x*(a)  ,  where 
x*(a)  is  a  solution  of  P[e(a)]  for  some  fixed  value  of  a  =  a  in 
10,1]  . 

Kiacco  and  Kyparisis  are  exploring  the  following  approach.  Assume 
the  problem  functions  are  twice  continuously  differentiable.  Let 

y*  =  x*(a)  and  e  =  e(a)  .  Then  the  convexity  of  f  implies  that  for 

any  vector  z  EE11,  the  first  order  condition 

Af  i  f(y*  +  z  ,  e)  -  f(y*,c)  >  V^ffy*,^)  z  (6.1) 

holds.  A  second  order  inequality  also  holds.  Assuming  the  Karush-Kuhn- 
Tucker  conditions  hold  at  y*  with  multipliers  u*  and  w*  .  Taylor's 

series  yields 

L*  ~  L(y*  +  z,  u*,  w*,e)  *  f(y*,  e)  +  1/2  zC\/^2L(n,  u*,  w*,  t)  z 
where  n  =  y*  +  3z  for  some  gc[0,l]  . 

If  y*  +  z  is  a  feasible  point  of  P(c)  ,  then  f(y*  +  z,  c)  >  L* 
and  hence  we  may  conclude  that 

Af  f(y*  +  z,  t)  -  f  (y*,  C)  >  1/2  ztVx2L(n,  u*,  w* ,  c.)  z  (6.2) 

ior  every  z  such  that  y*  +  z  e  R(e)  . 

We  are  seeking  a  bound  on  |  z  |  1  |  x*(a)  -  x*(a)[ 

Note  that  if  z  =  z  1  x*(d)  -  y*  ,  then  y*  +  z  =  x*(a)  c  R(e) 

as  observed  and  (6.1)  and  (6.2)  apply.  Further,  we  can  calculate  an 
overestimation  of  the  left-hand  side  of  these  inequalities  from  our 
previous  optimal  value  bounds  calculations,  obtaining  in  particular 
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that 

B(a)  x  f[x*(a),  e]  -  max  [ (ot ) ,  £2(ci)]  >  f[x*(u),  c]  -  f[x*(a),  c]  (6.3) 

_  f (y*  +  z,  <)  -  f(y*.  e) 

We  now  have  from  (6.1),  (6.2)  and  (6.3)  that 

Vxf(y*t  e)  z  <  B (a)  (6.4) 

1/2  zCVx2L(n,  u*.  w*,  e)  z  <  B(af  (6.5) 

It  remains  to  underestimate  the  left-hand  sides  of  (6.4)  and  (6.5) 
in  terms  of  j  z  |  .  A  natural  approach  is  to  seek  y^  >  0  and  y2  >  0 

such  that 


Vxf(y*.  c)  z  >  y1  j  z  |  (6.6) 

and 

1/2  z£Vx2L(n,  u*,  w*,  e)  z  >  y2  j  z  | 2  (6.7) 

Having  calculated  y^  and  y2  ,  we  could  then  conclude  from  (6.4)  and 
(6.6)  that  |  z  j  <  B(a)/y^  and  from  (6.5)  and  (6.7)  that  j  z  |  < 

B(a)/y2  . 

However,  noting  that  (6.6)  does  not  account  for  second  order  eflects 
and  that  the  first  order  information  yielding  (6.6)  is  not  represented 

2 

in  (6.7)  (e.g.,  for  a  linear  program  7  L  =  0  so  y2  =  0  and  (6.7) 

yields  no  bound),  suggests  combining  (6.1)  and  (6.2)  to  yield  a  bound 
incorporating  both  first  and  second  order  information  at  the  outset  and  a 
therefore  hopefully  more  comprehensive  and  sharper  bound.  We  note  first 


A 


24  - 


T-451 


that  simply  adding  (b.l)  and  (<>.2)  docs  not  load  to  a  hound  of  t  lu'  typo 
given  in  (6,6)  or  (b.7)  because  of  the  lack  of  homogonity  with  respect  to 
z  of  the  terms  involving  z  .  However,  a  simple  modification  does. 

Note  that  the  right-hand  side  of  (6.6)  is  nonnegative  if  y*  +  z  f  R(c)  , 

since  y*  solves  P(e)  .  We  may  therefore  square  both  sides  of  (6.1) 
and  add  the  result  to  (6.2),  weighted  respectively  by  u  >  0  and  3  >  0  , 

where  a  +  3  =  1  ,  for  additional  flexibility,  obtaining  for  all  z  such 
that  y*  +  z  t  R(£)  ,  aAf2  +  3Af  >  «(?  f(y*,  £)z)2  +  ~  ztv\(n>  u*, 
w*,  C)  z  . 

As  before,  this  expression  and  (6.3)  imply  that 

a(vx£(y*>  e)z)2  +  — ^ —  z  Vx2L(n,  u*,  w*,  e)  z  <  aB2(a)  +  is  B(a) 


If  we  can  find  any  y(ot,  3)  >  0  such  that 

-  \2 


M 


i  w^v/(y*.£) 


3 


~ 2l (n »  u*.  w*.  e)  — —  (6.8) 

—  i  X  i  ■“  i 


then  we  can  conclude  that 


z  |  2  <  aB2(a)  -f  8  B(a ) 


(6.9) 


The  parameters  a  and  3  have  been  introduced  for  additional 
flexibility  and  generality,  incorporating  the  bounds  based  on  (6.6)  and 
(6.7)  as  special  cases  of  (6.9)  and  allowing  the  possibility  of  improving 
on  the  former  bounds.  Ideally,  we  would  like  to  fix  x  and  3  such  that 
the  largest  value  of  y(a,3)  will  result,  and  the  theoretical  implications 
of  this  optimal  selection  might  be  interesting  to  explore.  In  practice,  a 
possible  strategy  might  be  to  first  check  the  bounds  obtained  from 
(a, 3)  *  (1,0)  ,  (0,1)  .  Obviously,  if  a  ■  1  and  3*0  then  y  *  y^ 

and  if  a  ■  0  and  3*1,  then  y  «  y^  .  Suppose  we  allow  at  roost  one 
attempt  to  improve  the  bound.  If  either  y^  or  y^  are  0  ,  but  not  both, 
we  could  accept  the  nonzero  value  for  y  to  calculate  the  bound.  If 
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=  U2  “  0  ’  We  misht  try  "unbiased"  weights,  a  =  3=1.  Finally, 

if  h1  >  0  and  \i^  >  0  »  we  ®ight  select  a  and  3  proportionate  to 
anc*  ^2  »  respectively,  e.g.,  we  may  take  a  =  ^/(u^  +  and  3  * 

+  ^2^  *  The  var:ious  possibilities  and  their  implications  will  be 
explored. 


Having  selected  values  for  u  and  3  ,  we  are  led  to  the  study 
of  a  problem  of  the  form 


min  ah  f— 5 - \2  + 

(x,u,w,z)  'X  I  z  I  '  ' 


V  2L  — 5- 
z  I  x  |  z 


over  an  appropriate  set  containing  (y*,  u*,  w*,  z)  .  The  calculation 
of  U  leads  to  an  analysis  of  various  nonlinear  programs  of  the  general 
form 


min  a(^f(x,e)z)2  +  -| —  zVl(x,  u*,  w*.  1)  z 
(x,z) 


P(a,3) 


s.t.  vx%fx»  e)  z  >  0  i  e{i|gi(x,e)  =  0} 


Vxhj(x,  e)  z  =  0 


z  =1 


with  possibly  some  additional  restrictions.  As  noted,  the  development 
of  bounds  results  using  this  approach  is  currently  being  investigated. 

Two  intriguing  conditions  associated  with  the  P(a,3)  bound  are 
noted,  that,  incidentally,  do  not  require  the  convexity  of  P(e)  . 

First,  if  the  Karush-Kuhn-Tucker  conditions  for  P(E)  hold  at 

A  A 

(x,  u*,  w*),  the  resulting  optimal  value  jj  of  P(a,3)  with  a  >  0  , 

3  >  0  is  positive  if  and  only  if  the  second  order  sufficient  conditions 
for  a  strict  local  minimum  of  P(e)  hold  at  (x,  u*,  w*)  .  If  P(e)  is 
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convex  in  x  or  if  y*  is  the  unique  local  solution  of  P(c)  associated 

A  A 

with  (u*,  w*),  then  x  =  y*  .  In  any  event,  x  would  be  a  local  strict 
minimum  of  P(t)  with  optimal  Lagrange  multipliers  u*,  w*  .  (This  is 
readily  verified  using  the  equivalent  form  of  the  second  order  conditions 
noted  by  Han  and  Mangasarian  (1979). 

The  second  fact  is  the  similarity  of  P(u,b)  with  the  quadratic 
subproblem  of  the  form 

min  V  f(x°  ,  e)  z  +  1/2  zCV  ^L(x°,  u°,  w°,  e)  z 

Z  X-  X 

s.t.  vxgi(x°,  e)  z  >  0  i  e  {i|g±(x°,  r)  =  0} 

Vh.(x°,  £)  z  =  0  j  =  l,..,,p 
x  j 

that  arises  in  connection  with  the  recursive  programming  methods  of 
Levitin  and  Polyak  (1966),  Robinson  (1972),  Wilson  (1963)  and  other 
contemporary  versions.  These  interesting  close  correspondences  with 
important  results  are  currently  being  investigated. 
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